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ENDOSCOPE SYSTEM 

e present application claims the benefit of Japanese 
Application No. Heisei 11-247588 filed in Japan on 
September 1, 1999, the contents of which are incorporated by- 
reference. 




!_=. BACKGROUND OF THE INVENTION 

o 

0 1 . Field of the Invention 

H The present invention relates to an endoscope system 

5j for assisting in objectively assessing the outline of a 
W section of an object, which lies at a desired cutting 

position, using image signals that represent images picked 
up by viewing the object from a plurality of viewing points. 
2 . Description of the Related Art 

In recent years, endoscopes have been widely employed 
in the fields of medicine and industries. Generally, an 
object appears planar in a view image picked up by an 
ordinary endoscope. It is therefore hard to identify the 
irregularity or the like in the surface of the object. 

Therefore, for example, Japanese Examined Patent 
Application Publication No. 8-12332 describes a measuring 
endoscope system for three-dimensionally measuring an object 
using images picked up by viewing the object from a 
plurality of viewing points. Moreover, Japanese Patent 
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Gazette No. 2778739 describes a measuring endoscope system 
that indicates a distance from a reference plane to a 
measuring point when the reference plane and measuring point 
are designated, enabling a user to objectively recognize the 
height or depth of a convex or concave part of an object. 

Conventional measuring endoscope systems can measure 
three coordinates that represent in three-dimensional space 
U a point designated in a reference image, a distance between 
Pi two points, or a depth from a designated plane. 
/J However, the conventional measuring endoscope systems 

|f request a user to designate a plurality of points, and infer 
^ the three-dimensional shape of an object of observation from 
2 three coordinates that represent each of the points. This 

5 

"3 poses a problem that it is hard to intuitively grasp the 

f 

Q three-dimensional outline of the object of observation. 

ru 

Moreover, the conventional measuring endoscope systems 
may adopt a method of acquiring three-dimensional 
information concerning an entire imaging range from image 
information so as to create a three-dimensional model. 
However, this poses a problem that it takes too much time 
for a computer to perform arithmetic operations required to 
create the three-dimensional model, making the method is 
impractical. 

Therefore, when the depth of an object, for example, a 
corroded portion of the inner wall of a pipe must be 
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measured, it takes much time to identify the deepest point. 

SUMMARY OF THE INVENTION 

An object of the present invention is to provide an 
endoscope system that acquires section information 
concerning a viewer's desired cutting plane when the desired 
cutting plane is designated using a displayed reference 
image . 

In short, an object of the present invention is to 
provide an endoscope system that makes it easy to designate 
a cutting plane whose measurement is desired, and assists in 
grasping the outline of a section of an object of 
observation, that is, the outline of the designated cutting 
plane. 

According to the present invention, there is provided 
an endoscope system having an imaging unit that picks up 
images of an object of observation while viewing the object 
of observation from a plurality of viewing points. Image 
signals representing the images, which the imaging unit has 
picked up while viewing the object of observation from the 
viewing points, are subjected to image processing and 
arithmetic operations, whereby stereo measurement is 
achieved . 

The endoscope system consists mainly of a corrected 
image producing means, an image displaying means, a cutting- 
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plane reference line designating means, a corresponding 
point searching means, a section information arithmetic 
means, and a section information outputting means. Herein, 
the corrected image producing means adopts one of the images 
picked up by viewing the object of observation from the 
plurality of viewing points as a reference image, and 
regards the other image as a comparison image. The 

m corrected image producing means corrects optical 

O 



Q 



deformations in the reference image and comparison image so 
as to produce a corrected reference image and corrected 
comparison image. Among the reference image, comparison 
images, corrected reference image, and corrected reference 
images, the image displaying means displays at least the 
reference image or corrected reference image on a screen. 
The cutting-plane reference line designating means is used 
to draw a cutting-plane reference line, which specifies a 
cutting-plane position that determines a section of the 
object of observation whose section information should be 
acquired, in an image displayed on the screen. The 
corresponding point searching means regards a point on the 
reference cutting-plane line in the corrected reference 
image as a point of attention, and searches the corrected 
comparison image for a corresponding point that is 
associated with the point of attention. The section 
information arithmetic means calculates three coordinates, 
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which represent a point in three-dimensional space whose 
mapping results in a point of attention on the cutting 
reference line according to the principles of 
trigonometrical measurement, using the position of the point 
of attention in the corrected reference image and the 
position of the corresponding point in the corrected 
comparison image searched for by the corresponding point 

M . searching means. The section information arithmetic means 

pis 

S thus acquires section information concerning the section of 

H the object of observation determined with the cutting-plane 

H position. The section information outputting means provides 

m the section information using values calculated by the 
section information arithmetic means. 

D The above and other objects of the present invention, 

•P. 

O and the features and advantages thereof will be more clearly 

m 

understood from the subsequent description made in reference 
to the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 to Fig. 15 are concerned with an embodiment of 
the present invention; 

Fig. 1 is an explanatory diagram schematically showing 
the configuration of an endoscope system capable of 
performing measurement; 

Fig. 2 is an explanatory diagram for explaining an 
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imaging unit incorporated in the distal portion of an 
inserting section of an endoscope and a measuring device 
having an arithmetic unit; 

Fig. 3 shows a scene where an object-of -observation 
region is observed using an endoscope; 

Fig. 4 is a flowchart describing stereo measurement; 

Fig. 5 shows an example of an endoscopic image and a 
. , cutting-plane reference line displayed on the screen of a 
monitor by means of a reference image displaying means; 

Fig. 6 is an explanatory diagram showing a cutting 
C| plane and a sectional contour line; 

W Fig. 7 is an explanatory diagram showing another 

O example of a reference line; 

O Fig. 8 shows an example of a sectional contour line 

Q displayed as section information; 

m 

Fig. 9 is an explanatory diagram showing an example of 
a sectional contour line displayed while being superposed on 
a reference image; 

Fig. 10 is an explanatory diagram showing a camera 
coordinate system defined in order to present section 
information; 

Fig. 11 shows another example of a sectional contour 
line displayed as section information; 

Fig. 12 shows another example of a sectional contour 
line displayed as section information; 



Fig. 13 shows still another example of a sectional 
contour line displayed as section information; 

Fig. 14 is a flowchart describing an algorithm for 
searching for a corresponding point; 

Fig. 15 is an explanatory diagram concerning monitor 
screen images displayed after searching for a corresponding 
point is completed; 

Fig. 16 to Fig. 22 are explanatory diagrams concerning 
jg the approaches to reduce ambiguousness in association 

^ between a plurality of images; 

M 

>•"- Fig. 16 is an explanatory diagram showing images 

W displayed on the screen of a monitor; 

Q Fig. 17 is a flowchart describing a method of improving 

m 

Q precision in section information by utilizing the order of 

T 

q points juxtaposed along a reference line; 



m 



Fig. 18 is a flowchart describing a method of improving 
precision in section information by utilizing the depths of 
surrounding points; 

Fig. 19 is a flowchart describing a method of improving 
precision in section information by utilizing a back- 
corresponding point; 

Fig. 20 is a flowchart describing a method of improving 
precision in section information by utilizing the magnitude 
of deviation from an epi-polar line; 

Fig. 21 is a flowchart describing a method of improving 
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precision in section information by utilizing a differential 
coefficient; 

Fig. 22 shows the shapes of small domains each defined 
to contain a point of attention; 

Fig. 23 is an explanatory diagram showing an example of 
a reference line defined by designating one point; and 

Fig. 24 is an explanatory diagram showing another 
example of a reference line defined by designating one point. 

h* 
O 
O 

p DESCRIPTION OF THE PREFERRED EMBODIMENTS 

SO Embodiments of the present invention will be described 

m 

UJ with reference to the drawings below. 

Q Fig. 1 to Fig. 15 are concerned with an embodiment of 

Q the present invention. Fig. 1 is an explanatory diagram 

p schematically showing the configuration of an endoscope 



ft! 



system capable of performing measurement. Fig. 2 is an 
explanatory diagram showing an imaging unit incorporated in 
the distal portion of an inserting section of an endoscope 
and a measuring device having an arithmetic unit. Fig. 3 
shows a scene where an object-of -observation region is 
observed using an endoscope. Fig. 4 is a flowchart 
describing stereo measurement. Fig. 5 shows an example of 
an endoscopic image and a cutting-plane reference line 
displayed on the screen of a monitor by means of an image 
displaying means. Fig. 6 is an explanatory diagram 
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concerning a cutting plane and a sectional contour line. 
Fig. 7 shows another example of a reference line. Fig. 8 
shows an example of a sectional contour line displayed as 
section information. Fig. 9 is an explanatory diagram 
showing an example of a sectional contour line displayed 
while being superposed on a reference image. Fig. 10 is an 
explanatory diagram showing a camera coordinate system 
defined for presenting section information. Fig. 11 shows 

O another example of a sectional contour line displayed as 

P 

C* section information. Fig. 12 shows another example of a 

00 sectional contour line displayed as section information, 

ijj Fig. 13 shows still another example of a sectional contour 

p line displayed as section information. Fig. 14 is a 

m 

p flowchart for explaining an algorithm for searching for a 
.Q corresponding point. Fig. 15 is an explanatory diagram 



showing screen images displayed on the monitor after 
searching for a corresponding point is completed. 

As shown in Fig. 1, an endoscope system 1 for 
performing measurement according to the present embodiment 
consists mainly of an electronic endoscope 2 (hereinafter 
simply an endoscope) and a measuring device 3. An imaging 
unit for converging view images of an object-of -observation 
region on an imaging device that will be described later is 
incorporated in the endoscope 2 . The measuring device 3 
includes an arithmetic unit that performs image processing 
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on image signals which represent the view images and are 
produced by the endoscope 2, and that performs various 
measurements using image data resulting from the image 
processing. 

Incidentally, a light source unit (not shown) for 
supplying illumination light to an object of observation is 
incorporated in the measuring device 3 . Endoscopic images 
of an ob ject-of-observation region are displayed on the 

!f 

C5 screen of a monitor 30. 

O 

Cf The endoscope 2 has an elongated inserting section 20. 

§** 

CO An operating section 21 serving as a hand-held section is 

y3 

Ul formed proximally to the inserting section 20. A universal 
Q cord 23 having a connector 22, which is freely attachable or 

C°J detachable to or from the measuring device 3, fixed to the 

IT 

p proximal end thereof is extended from the lateral surface of 
the operating section 21. 

The inserting section 20 includes a distal portion 24, 
a bending portion 25, and a flexible portion 26 that are 
juxtaposed in that order from the distal end of the 
endoscope. An imaging optical system that will be described 
later is incorporated in the distal portion 24. The bending 
portion 25 has a plurality of bending pieces concatenated so 
that the pieces can revolve freely. The flexible portion 26 
is formed with an elongated soft member. Note that a 
numeral 27 is referred to an operating knob 27 which is used 
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to bend the bending portion 25. 

As shown in Fig. 2, an imaging unit 40 composed of a 
pair of objective lens systems 41 and 42 and an imaging 
device 43 is incorporated in the distal portion 24. The 
imaging device 43 photoelectrically converts optical images, 
which are converged on the imaging surface thereof after 
passing through the objective lens systems 41 and 42, into 
image signals. 

y. 

P The inclusion of the objective lens systems 41 and 42 

O 

=0 is intended to image an object-of-observation region while 

N- 

p viewing it from a plurality of viewing points. Optical 

m 

Ly images passing through the objective lens systems 41 and 42 

q are converged on the imaging surface of the imaging device 

01 

pi 43. In other words, the endoscope 2 employed in the present 

S embodiment is a so-called stereoscopic endoscope that 

tea' 

produces stereo images that cause a parallax. 

As shown in Fig. 1 and Fig. 2, the measuring device 3 
consists mainly of a camera control unit (hereinafter 
abbreviated as CCU) 31, a video capture circuit 32, a host 
computer 33, and a console 34. The CCU 31 converts image 
signals, which result from photoelectric conversion 
performed by the imaging device 43, into video signals 
according to which images are displayed on the monitor 30. 
The video capture circuit 32 converts the video signals, 
which are produced by the CCU 31, into digital image signals. 
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The host computer 33 serving as an arithmetic unit uses the 
digital image signals produced by the video capture circuit 
32 to perform arithmetic operations required for measurement. 
The console 34 is used to operate the measuring device 3 via 
the host computer 33. 

As shown in Fig. 3, illumination light generated by the 
light source unit incorporated in the measuring device 3 is 
passed through the connector 22, and propagated over the 
universal cable 23 and light-guide optical fibers (not 
shown) that run through the inserting section 20. The 
illumination light is then emitted towards an object-of- 
observation region 4 through illumination windows 44. 
Consequently, endoscopic images of the ob ject-of-observation 
region 4 are displayed on the monitor 3 0 of the measuring 
device 3. 

Incidentally, according to the present embodiment, the 
console 34 is independent of a main unit 35 of the measuring 
device, and the monitor 30 is integrated with the main unit 
35 of the measuring device. Depending on a purpose of use, 
however, the console 34 may respectively be integrated with 
or independent of the main unit 35 of the measuring device. 

Referring to Fig. 4 to Fig. 24, actions to be performed 
and operations to be exerted when the endoscope system 1 
having the foregoing components is used to perform stereo 
measurement will be described below. 
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For example, assume that the inserting section 20 of 
the endoscope 2 is inserted into a pipe, and the observation 
of the inside of the pipe is started to see if there is a 
corroded part or a flaw. 

First, as shown in step SI 01 described in the flowchart 
of Fig. 4, view images of the object-of -observation region 4 
illuminated by the illumination light are passed through the 
objective lens systems 41 and 42, and converged on the 
O imaging surface of the imaging device 43 respectively. The 

o 

J3 view images on the imaging device 43 are photoelectrically 

03 converted into image signals. The respective image signals 

y are transmitted to the CCU 31 incorporated in the measuring 

Ss 

Q device 3 through the connector 22 over signal lines. The 

m 

.q signal lines that are not shown are run through the 

q inserting section 20, operating section 21, and universal 

lU cable 23. 

The CCU 31 having received the image signals produces 
video signals. The video signals are transferred to the 
video capture circuit 32 and converted into digital image 
signals. As shown in step S102, the digital video signals 
are transferred to the host computer 33, while the image 
signals are transferred to the monitor 30, whereby the 
endoscopic view images are displayed on the screen of the 
monitor 30, as shown in step SI 03. 

If a corroded part or the like is discovered during 
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viewing of the monitor, the corroded part is measured. At 
this time, a viewer first uses the console 34 to select a 
stereo measurement mode. This causes the image displaying 
means incorporated in the host computer 33 to separately 
display a corrected reference image 50 and a corrected 
comparison image 51 on the monitor 30, as shown in Fig. 5. 
At this time, one of the endoscopic images of the view 
images picked up by the objective lens systems 41 and 42, 

if 

O for example, the image picked up by the objective lens 

G 

C! system 41 is adopted as a reference image. The other image 

|=53 

[ill picked up by the other objective lens system 42 is regarded 

m 

yj as a comparison image. 

C| Incidentally, the images 50 and 51 are produced as the 

p corrected images to be used for measurement by means of a 
m corrected image producing means as shown in step S104. 
! ™ Specifically, the digital image signals fetched into the 

host computer 33 have distortions thereof corrected using a 
distortion correction coefficient pre-set in the imaging 
unit 40. The corrected reference image 50 and corrected 
comparison image 51 that are the corrected images to be used 
for measurement are used to perform stereo measurement, that 
is, to search for a corresponding point. 

For measurement, as shown in step SI 05, two points A 
and B are firstly designated in the reference image 50 shown 
in Fig. 5 using a cutting-plane reference line designating 



- 15 - 



means in order to draw a cutting-plane line that specifies a 
cutting-plane position which determines a section whose 
section information must be acquired for measurement. 
Specifically, for the operation, a viewer uses the console 
34 or the like to move a pointer such as an arrow mark or a 
cursor that is displayed on the screen. 

When the viewer designates the points A and B, a 
straight line passing through the points A and B as 
O indicated with a dashed line in Fig. 5 is regarded as a 
?J3 cutting-plane reference line (hereinafter abbreviated as a 
00 reference line) 53. Otherwise, a segment linking the points 
|ij A and B as indicated with a solid line therein is regarded 
Q as a cutting-plain reference segment (hereinafter 

p abbreviated as a reference segment) 54. A cutting plane 

1 

m defined with the cutting-plane reference line 53 will be 
described in conjunction with Fig. 6. The image of the 
object of observation 4 is, in principle, converged as an 
inverted image opposite to the object of observation with an 
optical center between them. In the drawing, for a better 
understanding, the corrected reference image 50 is depicted 
as an erect image between the object of observation 4 and 
the optical center L. 

Incidentally, aside from the reference line 53 drawn as 
mentioned above, another straight line indicated with a dot- 
dash line in Fig. 7 may be drawn as an auxiliary line 55 
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that is another reference line. The auxiliary line 55 meets 
the reference line 53 at a certain angle (crosses the 
reference line 53 at right angles in the drawing) at a point 
interposed between the points A and B (middle point in the 
drawing ) . 

A viewer's intended cutting plane 56 is a plane 
indicated with an alternate long and two short dashes line 
and orthogonal to the corrected reference image 50 shown in 
the figure. The cutting plane 56 is defined as a plane 
containing the reference line 53 and the optical center L of 
the optical system that picks up the reference image. 

Points Al and Bl in the figure are mapped points on the 
surface of the ob ject-of -observation region 4 resulting from 
mapping of the points A and B in the corrected reference 
image 50. That is, a sectional contour line 57 that is an 
object of display in the present embodiment is defined as a 
line shared by the surface of the ob ject-of -observation 
region 4 seen from a viewing point and the cutting plane 56. 

Subsequently, the optical axis of the reference image 
optical system is projected as a point on the corrected 
reference image 50, and the point serves as the image center 
O of the corrected reference image 50. Incidentally, a 
numeral 58 is referred to an optical axis 58 which is shown 
as a dot-dash line linking the image center 0 and the 
optical center L in Fig. 6. 
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As shown in step S106, all pixels lying along the 
reference line 53 (or the reference segment 54) in the 
corrected reference image 50 that is a corrected image to be 
used for measurement are regarded as points of attention in 
order to acquire section information concerning a section 
determined with the cutting plane 56. Therefore, as shown 
in step S107, a corresponding point searching means searches 
for a corresponding point, that is, a pixel in the corrected 

|l reference image 51 associated with each point of attention. 

- 

<£? Consequently, a group of corresponding points 59 that will 

Js* 

il be described later is sampled. An algorithm for searching 

Jij for a corresponding point will be described later in 

O conjunction with a flowchart in Fig. 14 and Fig. 15. 

f- If corresponding points are found at step S107 of 

|j corresponding point search, control is passed to step S108. 

m 

A difference between the position of each point of attention 
in the corrected reference image 50 and the position of its 
corresponding point in the corrected comparison image 51 is 
calculated, that is, a parallax of each point of attention 
relative to its corresponding point is calculated. Control 
is then passed to step S109. Based on the calculated 
parallax and optical data, a section information arithmetic 
means calculates three coordinates representing a point in 
three-dimensional space that results from mapping of each 
point of attention. The optical data includes a base length 
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that is a distance between the optical centers of the 
optical systems and is calculated in advance, the focal 
lengths relevant to the optical systems, and coordinates 
representing each of mapped points in the corrected image to 
be used for measurement resulting from mapping of the 
optical axes of the optical systems. 

If three coordinates representing each point, that is, 
all pixels lying along the reference line 53 are calculated 
O at step S109, control is passed to step SI 10. Based on the 

o . 

*fl three coordinates of each point, section information 

U 

t$ concerning a section determined with the cutting plane 56 is 

'# 

fxj constructed in a viewer's desired manner by selecting any of 
P four display forms described later. Control is then passed 

q to step Sill. A section information outputting means 

M 

p displays, as a contour line outlining a section, section 

S3' 

information that assists in readily grasping the outline of 
a section of an object of observation on the screen of the 
monitor 30. Incidentally, when the contour line outlining a 
section is displayed on the screen of the monitor 30, only 
the contour line may be displayed on behalf of the corrected 
reference image 51 on the screen. Otherwise, another window 
may be displayed on the screen and the contour line may be 
displayed within the window. Otherwise, the contour line 
may be superposed on the corrected reference image 50. 
Now, the section information and the algorithm for 
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searching for a corresponding point will be described below. 

To begin with, section information will be described. 
The section information may be displayed in any of four 
display forms described below. 

( 1 ) Directly displaying section information concerning 
a section determined with a cutting plane 

Section information is displayed as a contour line 57a 
outlining a section as shown in Fig. 8. In the figure, the 
O axis of ordinates represents an orthogonal projection of the 

r- 

C= reference segment 54 or reference line 53 on the cutting 
© plane 56, and the axis of abscissas represents an orthogonal 
W projection of the optical axis 58 on the cutting plane 56. 
Q In this case, the contour line outlining the section is 
jp rendered linearly. 

■q In order to display the contour line 57a outlining the 

fit 

,fe section on the screen of the monitor 30 during measurement, 
another window may be opened on the monitor and the contour 
line 57a outlining the section may be displayed within the 
window. Otherwise, the contour line 57a outlining the 
section may be, as shown in Fig. 9, superposed on the 
reference image 50 displayed on the monitor 30. 
Incidentally, when the contour line is superposed on the 
reference image, an arrow mark C or the like is displayed as 
a mark indicating the direction of a viewing point. The 
mark helps learn the irregularity of the contour line. 
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(2) Projecting section information on a plane in space 
As shown in Fig. 10, a camera coordinate system is 

defined as a coordinate system having an origin thereof 
aligned with the optical axis L of the optical system that 
picks up the reference image 50. Moreover, the x axis of 
the coordinate system is extended horizontally rightwards 
towards the ob ject-of-observation surface 52, the y axis 
thereof is extended vertically upwards, and the z axis 
thereof is extended in a depth direction. Section 
information is projected on the xz plane shown in Fig. 11(a) 
or the yz plane shown in Fig. 11(b), whereby a contour line 
57b or 57c outlining the section is displayed. In this 
display form, the same projection technique as the one 
employed in plotting a normal projection drawing can be 
adopted. It is therefore advantageous that a viewer 
accustomed to the projection technique for plotting a normal 
projection drawing can easily grasp the outline of a section. 
Note that coordinate axes or a plane of projection can be 
defined arbitrarily depending on a purpose of use. 

(3) Displaying section information quasi three- 
dimensionally 

As shown in Fig. 12, a viewing point other than a 
viewing point from which a line of sight extends through the 
endoscope 2 is newly set in real space. A contour line 57d 
outlining a section seen from the set viewing point is 



displayed. A cylindrical graphic in the drawing represents 
the position of the endoscope 2 in the space for better 
comparison. Moreover, an image three-dimensionally 
expressing the position of the viewing point may be 
simultaneously displayed on another screen. In this display 
form, the viewing point is shifted sequentially in order to 
change the apparent position of a section. Consequently, 
the three-dimensional outline of an object of observation 
can be grasped intuitively. 

(4) Displaying section information by sampling a depth 
that is a distance from a reference line defined in real 
space 

As shown in Fig. 8 and Fig. 13, a distance from a point 
Dl to a point D2 is indicated with a line perpendicular to 
the reference segment 54 in the corrected reference image 50. 
The point Dl is a point on the surface of the object of 
observation 4 resulting from mapping of the point D on the 
reference segment 54 or reference line 54. The point D2 is 
a point on a line A1B1 linking points Al and Bl, which are 
points in real space resulting from mapping of the points A 
and B, resulting from mapping of the point D. The point D 
is set to all points lying along the reference segment 54 or 
reference line 53, whereby a contour line 57e outlining a 
section is displayed. According to this displaying 
technique, as described in the paragraph (1), an arrow mark 



C indicating the direction of a viewing point is displayed 
as a cue to learn the irregularity in the outline of a 
section. Incidentally, the sectional contour line 57e may 
be colored depending on the length of the segment D1D2 in 
order to provide the cue. 

This displaying technique can compactly display a 
result even with a contour line whose depth varies 
outstandingly . 

Next, the algorithm for searching for a corresponding 
point will be described below. 

Searching for a corresponding point is always performed 
in a corrected image to be used for measurement. Searching 
for a corresponding point is known as template matching or 
window matching, where an algorithm for searching for a 
corresponding point using information of points surrounding 
a point of attention is employed. 

A straight line in space that is projected as a point 
on the corrected reference image 50 is projected as a 
straight line on the corrected comparison image 51. The 
straight line is referred to as an epi-polar line. A 
corresponding point in the corrected comparison image 51 
associated with a point of attention in the corrected 
reference image 50 theoretically lies only on the epi-polar 
line. Therefore, searching for a corresponding point should 
be performed on a domain containing the epi-polar line and 
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several pixels above and below the epi-polar line in 
consideration of an error. Therefore, a searching procedure 
is described in the flowchart of Fig. 14. 

First, searching for a corresponding point is started 
as shown in step SI 07. Consequently, a domain Pi is defined 
to contain a point of attention as shown in step S201. 
Control is then passed to step S202 where an epi-polar line 
relevant to the point of attention is calculated. Points 
belonging to a domain that contains the epi-polar line and 
several pixels above and below the epi-polar line are 
g| sampled as candidates for a corresponding point, 
juj Next, control is passed to step S204 where a domain P2 

q whose size is the same as the size of the domain PI 
g containing the point of attention is defined to contain each 
candidate. A coefficient of mutual correlation between the 
normalized powers of the domain PI containing the point of 
attention and the domain P2 containing each candidate for a 
corresponding point is calculated or the square of the 
difference between the powers is calculated. Thus, the 
degree of correspondence by which the candidate corresponds 
to the point of attention is calculated. As shown in steps 
S206 and S207, the degrees of correspondence of all the 
candidates are calculated, and the calculated degrees of 
correspondence are preserved in relation to all the 
candidates. After preserving the calculated degrees of 



P 

m 
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correspondence of all the candidates is completed, the 
candidate exhibiting the highest degree of correspondence is 
adopted as a corresponding point associated with the point 
of attention as shown in steps S208 and S209. If it is 
judged at step S210 that the corresponding points associated 
with all the candidates have been searched for, 
corresponding point search is completed. Control is then 
passed to step S108. 

i=* 

O Through the corresponding point search, a corresponding 

Q 

p point associated with each of the points of attention lying 
IS along the reference segment 54 in the corrected reference 

m 

W image 50 are sampled as described in Fig. 14. When the 
fp corresponding points associated with all the points of 

jjj attention have been searched for, the group of corresponding 

.«$ 

P points 59 that is a set of corresponding points is displayed 

ill - 

on the corrected comparison image 51. 

As described above, a cutting-plane reference line can 
be drawn in order to specify the position of a section a 
viewer wants to observe, by selecting a stereo measurement 
mode and designating two points in a reference image 
displayed on the screen of the monitor during observation. 

Moreover, the viewer can easily grasp the outline of 
the section by displaying section information concerning a 
section determined with the cutting-plane reference line 
drawn by a viewer as a contour line outlining the section on 
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the screen of the monitor in the viewer's desired display 
form. 

This enables the viewer to swiftly specify an observed 
portion, to intuitively grasp the outline of a section of 
the specified observed portion and to shortly complete 
observation in case a corroded part or the like is found 
during observation. 

When a range of points within which a corresponding 
2 point is searched for may be limited to a cutting-plane 
*P reference segment but not to a cutting-plane reference line. 

w This leads to a further reduction in the time required to 

P. 

Ui complete searching for a corresponding point. 

S Moreover, according to the present embodiment, 

m 

O corresponding points associated with all points of attention 

£i 

Q lying along the reference line 53 or reference segment 54 

fit 

are searched for. It is essentially ambiguous to how the 
reference image 50 is associated with the comparison image 
51. It is therefore hard to associate a point within an 
occlusive domain or a point adjoining the occlusive domain 
with another point. The occlusive domain refers to a domain 
whose power has no difference from that of a surrounding 
domain, a domain containing pixels that are visible in the 
reference image but invisible in the comparison image, or a 
domain containing pixels that are invisible in the reference 
image but visible in the comparison image. 
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Consequently, there is a fear of taking an incorrect 
point for a corresponding point. If an incorrect point is 
taken for a corresponding point, a noise is contained in 
position information concerning the group of corresponding 
points 59 to be displayed- In some cases, the outline of an 
object of observation may be judged incorrectly. 

Therefore, a plurality of approaches that will be 
described below has been devised in efforts to exclude a 
point that may be associated with an incorrect one or to 
improve precision in association. Consequently, precision 
in acquired section information can be improved. 

A description will be made of the approaches to reduce 
ambiguousness in associating one point in one image with 
another point in another image. 

(1) A method of improving precision in section 
information by utilizing the order of points lying along a 
reference line will be described with reference to the 
flowcharts of Fig. 16 and Fig. 17. 

As described in the flowcharts of Fig. 16 and Fig. 17, 
at step S301, the order of all points of attention lying 
along the reference line 53 is detected. The order shall 
not change even if the reference line 53 is projected onto 
an axis orthogonal to a base line and parallel to the 
imaging surface of the imaging device. The base line is a 
straight line linking the optical center of the optical 



system that picks up the reference image and the optical 
center of the optical system that picks up the comparison 
image. The projection is the same as the projection of the 
reference line 53 onto an axis 62 in the reference image 51 
that is orthogonal to an epi-polar line 61 corresponding to 
the image center 0 of the reference image 50. 

Consequently as shown in step S302, projected points on 
the axis 62 onto which the corresponding points associated 
with all points of attention are projected are detected. 
Control is then passed to step S303. The order of the 
projected points on the axis 62 is then detected. 

As shown in steps S304, S3 05, and S306, it is judged 
whether the order of the points of attention is inconsistent 
with the order of the projected points of the corresponding 
points. A point of attention whose order is inconsistent 
with the order of the projected point of the corresponding 
point thereof is regarded as a point that has been 
unsuccessfully associated with another and then disregarded. 
In contrast, a point of attention whose order is consistent 
with the order of the projected point of the corresponding 
point thereof is adopted. In other words, for example, 
assume that points of attention a, b, c, d, and e are 
juxtaposed in that order along the reference line. If the 
projected points of the corresponding points are juxtaposed 
in order of a, b, d, c, and e, the points of attention d and 



c are disregarded because their orders are inconsistent with 
the orders of the projected points of the corresponding 
points thereof. 

The above processing is performed as a step interposed 
between steps S106 and S107 in the flowchart of Fig. 4. 
Step S107 and subsequent steps are performed only on adopted 
points . 

(2) A method of improving precision in section 
information by utilizing depths of surrounding points will 
be described with reference to the flowchart of Fig. 18. 

First, a range of points from which points of attention 
are sampled at step S106 is extended to contain pixels 
adjoining the points of attention. In other words, as shown 
in step S401 in Fig. 18, the total of nine points including 
a point of attention on the reference line 53 and eight 
surrounding points are associated with other points. 

As shown in steps S402 and S403, the average of three 
intermediate depths (other than three largest depths and 
three smallest depths) among the depths of the nine points, 
that is, a median is adopted as the depth of the point of 
attention. This processing is a kind of low-pass filtering 
and is performed as a step interposed between step S109 and 
step SI 10 in the flowchart of Fig. 4. 

(3) A method of improving precision in section 
information by utilizing back-corresponding points will be 
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described with reference to the flowchart of Fig. 19. 

After a corresponding point associated with a certain 
point of attention in the corrected reference image 50 is 
detected in the corrected comparison image 51 at step S107, 
the corresponding point in the corrected comparison image 51 
is back-associated with a point in the corrected reference 
image 50 as shown in step S501 in Fig. 19 according to a 
sequence described in the flowchart of Fig. 14. 

As shown in steps S502, S503, S504, and S505, it is 
judged that the back-corresponding point corresponds to the 
original point of attention. A back-corresponding point 
that does not correspond to a point of attention is regarded 
as a point that has been unsuccessfully associated with 
another point and disregarded. A back-corresponding point 
that corresponds to a point of attention is adopted. This 
processing is performed as a step interposed between step 
S107 and step S108 in Fig. 4. Step S108 and subsequent 
steps are performed only on adopted points. 

(4) A method of improving precision in section 
information by utilizing the magnitude of deviation from an 
epi-polar line will be described with reference to the 
flowchart of Fig. 20. 

The deviation of a corresponding point in the corrected 
reference image 51, which results from association, from an 
epi-polar line can exhibit a certain trend. Based on this 



theory, the magnitudes of deviation of all corresponding 
points are detected as shown in step S601 described in Fig. 
20. Then as shown in step S602, the detected magnitudes of 
deviation are statistically processed. As shown in step 
S603, the trend of the magnitudes is inferred from the 
results of the statistical processing, and a threshold 
indicating a maximum permissible magnitude of deviation is 
determined . 

Thereafter, as shown in steps S604, S605, S606, and 
S607, a point of attention relevant to a corresponding point 
whose magnitude of deviation exceeds the threshold is 
regarded as a point that has been unsuccessfully associated 
with another and disregarded. A point of attention relevant 
to a corresponding point whose magnitude of deviation is 
equal to or smaller than the threshold is adopted. This 
processing is performed as a step interposed between step 
S107 and step S108 described in Fig. 4. Step S108 and 
subsequent steps are performed only on adopted points of 
attention. 

(5) A method of improving precision in section 
information by utilizing a differential coefficient will be 
described with reference to the flowchart of Fig. 21. 

Association is likely to succeed when the power of a 
domain containing a point of attention changes but is hard 
to succeed when the power thereof does not change. 



Therefore, points of attention are selected from a domain 
whose power changes. This leads to a reduction in 
ambiguousness in association. 

Namely, differential filtering is performed on a domain 
containing a point of attention in order to calculate a 
change in the power of the domain as shown in step S701 in 
Fig. 21. 

As shown in steps S702, S703, S704, and S705, it is 
judged whether the power change is equal to or larger than a 
threshold and thus detected whether the power has changed. 
If the power change is equal to or smaller than the 
threshold, the point of attention is disregarded. If the 
power change is equal to or larger than the threshold, the 
point of attention is adopted. In other words, only a 
corresponding point that is associated with a pixel 
contained in a domain whose power is judged to have changed 
is searched for. This processing is performed as a step 
interposed between step S106 and step S107 described in Fig. 
4. Step SI 07 and subsequent steps are performed only on 
adopted points of attention. 

(6) A method of improving precision in section 
information by adjusting the shape of a domain containing a 
point of attention will be described below. 

If a point of attention adjoins an occlusive domain, 
the occlusive domain may overlap a domain defined to contain 
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the point of attention and used for searching. This leads 
to ambiguous association. Therefore, small domains in each 
of which the point of attention lies on the perimeter 
thereof are defined, and a candidate for a corresponding 
point associated with the point of attention is searched for 
within each small domain. The degree of correspondence 
exhibited by the candidate is then detected. Consequently, 
the highest degree of correspondence exhibited by any 
candidate for the corresponding point is adopted as the 
degree of correspondence by which the corresponding point 
corresponds to the point of attention. Examples of small 
domains are shown by the right-hand side of a border line 
indicated with a dashed line in Fig. 22. Black dots in the 
drawing indicate points of attention. A normally defined 
domain is shown by the left-hand side of the border line. 

One of the aforesaid six methods is used or two or more 
of the aforesaid six methods are arbitrarily used in 
combination in order to acquire section information, whereby 
high-precision section information can be acquired. 

Moreover, instead of acquiring section information by 
designating two points as mentioned above, one point in the 
corrected reference screen image 5 OA may be adopted as a 
sole reference point 71 as shown in Fig. 23 and Fig. 24. A 
straight line passing through the reference point 71 may 
then be adopted as a reference line. Section information 
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may thus be acquired. 

Incidentally, the reference line may be defined as a 
line described below. 

(1) Line 72 shown in Fig. 23 , that is, a line passing 
through the designated reference point 71 and being parallel 
to the horizontal direction of the screen of the monitor 30 

(2) Line 73 shown in Fig. 23, that is, a line passing 
through the designated reference point 71 and being parallel 
to the vertical direction of the screen of the monitor 30 

(3) Lines 72 and 73, that is, lines passing through the 
designated reference point 71 and being parallel to the 
horizontal and vertical directions of the screen of the 
monitor 3 0 

(4) Line 74 shown in Fig. 24, that is, a line passing 
through the designated reference point 71 and meeting the 
two straight lines 72 and 73 at any angles 

Thus, one point is designated in order to define a 
reference line. This results in simpler operation. A place 
whose sectional outline must be inspected can be designated 
directly. 

Although an example using two images is explained in 
the present embodiment, three or more images may also 
perform a section display. 

Moreover, as the second embodiment, images to be 
displayed in a stereo measurement mode may be employed as a 
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reference image and a comparison image that are not 
corrected in place of the corrected reference image 50 and 
corrected comparison image 51. A reference line then may be 
drawn in the reference image. Consequently, production of 
corrected images to be used for measurement can be 
simplified. The time required to select the stereo 
measurement mode can be shortened drastically. Note that 
the other operations and advantages are identical to those 
of the first embodiment. 

Having described the preferred embodiments of the 
present invention by referring to the accompanying drawings, 
it should be understood that the present invention is not 
limited to the embodiments but a person skilled in the art 
can make various changes and modifications without departing 
from the spirit or scope of the invention defined in the 
appended claims . 



